Ultrasound imaging apparatus, method of controlling the same, and computer program product

ABSTRACT

An ultrasound imaging apparatus includes: a probe configured to transmit ultrasound waves and detect echo signals; a display; and at least one processor configured to generate an ultrasound image based on the echo signals, wherein the at least one processor is further configured to obtain a three-dimensional (3D) medical image of a uterus, determine a target position for embryo transfer based on the 3D medical image, obtain a real-time ultrasound image of the uterus, identify the target position in the real-time ultrasound image, and control the display to display the real-time ultrasound image and information about the target position.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application is based on and claims priority under 35 U.S.C. § 119to Korean Patent Application No. 10-2019-0027016, filed on Mar. 8, 2019,in the Korean Intellectual Property Office, the disclosure of which isincorporated by reference herein in its entirety.

BACKGROUND 1. Field

The disclosure relates to ultrasound imaging apparatuses, methods ofcontrolling the same, and computer program products.

2. Description of Related Art

Ultrasound imaging apparatuses transmit ultrasound signals generated bytransducers of a probe to an object and receive information aboutsignals reflected from the object, thereby obtaining at least oneultrasound image of an internal part, for example, soft tissue or bloodflow, of the object. Ultrasound imaging apparatuses are capable ofobtaining medical images in real-time. Due to this, such ultrasoundimaging apparatuses have been widely used to obtain real-time imagesduring medical procedures or surgeries.

SUMMARY

Provided are ultrasound imaging apparatuses and methods of controllingthe same, which are capable of increasing a success rate of embryotransfer to the uterus.

Provided are also ultrasound imaging apparatuses and methods ofcontrolling the same, which are capable of increasing user convenienceand providing a guide to the user during embryo transfer to the uterus.

Additional aspects will be set forth in part in the description whichfollows and, in part, will be apparent from the description, or may belearned by practice of the presented embodiments.

In accordance with an aspect of the disclosure, an ultrasound imagingapparatus includes: a probe configured to transmit ultrasound waves anddetect echo signals; a display; and at least one processor configured togenerate an ultrasound image based on the echo signals, wherein the atleast one processor is further configured to obtain a three-dimensional(3D) medical image of a uterus, determine a target position for embryotransfer based on the 3D medical image, obtain a real-time ultrasoundimage of the uterus, identify the target position in the real-timeultrasound image, and control the display to display the real-timeultrasound image and information about the target position.

the at least one processor may be further configured to acquire a valueof at least one parameter based on the 3D medical image, determine thetarget position based on the value of the at least one parameter, matchthe value of the at least one parameter acquired based on the 3D medicalimage to the real-time ultrasound image, and control the display todisplay information about the value of the at least one parameter in thereal-time ultrasound image.

The at least one parameter may include at least one of an endometrialthickness, a blood flow amount, a junction zone location, a uterineboundary location, an endometrial location, a myoma location, and anadenomyoma location, or a combination thereof.

The probe may include at least one sensor including at least one of aposition sensor, a gyro sensor, and an accelerometer, and the at leastone processor may be further configured to register the 3D medical imagewith the real-time ultrasound image based on a signal detected by the atleast one sensor.

The at least one processor may be further configured to provideinformation about a position of the real-time ultrasound image on the 3Dmedical image.

The 3D medical image and the real-time ultrasound image may berespectively obtained by apparatuses using different imaging modalities

The at least one processor may be further configured to provide, basedon the 3D medical image, information about the probability of successfulimplantation at a position of at least a portion of the uterus.

The at least one processor may be further configured to provideinformation about a distance between a plane corresponding to thereal-time ultrasound image and the target position.

The least one processor may be further configured to provide a guide formoving the probe to obtain the real-time ultrasound image correspondingto the target position.

The real-time ultrasound image may show at least one procedure toolinserted into the uterus, and

the at least one processor may be further configured to control thedisplay to display the 3D medical image and the real-time ultrasoundimage and provide, on the 3D ultrasound image, information about aposition of the at least one procedure tool detected in the real-timeultrasound image.

The real-time ultrasound image may correspond to a real-timetwo-dimensional (2D) ultrasound image or a real-time 3D ultrasoundimage.

In accordance with another aspect of the disclosure, a method ofcontrolling an ultrasound imaging apparatus includes: obtaining a 3Dmedical image of a uterus; determining a target position for embryotransfer based on the 3D medical image; obtaining a real-time ultrasoundimage of the uterus; identifying the target position in the real-timeultrasound image; and displaying the real-time ultrasound image andinformation about the target position.

In accordance with another aspect of the disclosure, a computer programis stored on a recording medium and includes at least one instructionthat, when executed by a processor, performs a method of controlling anultrasound imaging apparatus, the method including: obtaining a 3Dmedical image of a uterus; determining a target position for embryotransfer based on the 3D medical image; obtaining a real-time ultrasoundimage of the uterus; identifying the target position in the real-timeultrasound image; and displaying the real-time ultrasound image andinformation about the target position.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other aspects, features, and advantages of certainembodiments of the disclosure will be more apparent from the followingdescription taken in conjunction with the accompanying drawings, inwhich:

FIG. 1 is a block diagram illustrating an ultrasound diagnosis apparatusaccording to an exemplary embodiment;

FIGS. 2A, 2B, and 2C are diagrams respectively illustrating anultrasound diagnosis apparatus according to an exemplary embodiment;

FIG. 3 is a block diagram of a structure of an ultrasound imagingapparatus according to an embodiment;

FIG. 4 is a diagram for explaining a process of transferring an embryoto the uterus, according to an embodiment;

FIG. 5 is a flowchart of a method of controlling an ultrasound imagingapparatus, according to an embodiment;

FIG. 6 illustrates a three-dimensional (3D) medical image and atwo-dimensional (2D) ultrasound image according to an embodiment;

FIG. 7 illustrates a maximum implantation potential (MIP) pointaccording to an embodiment;

FIG. 8 illustrates a graphical user interface (GUI) view according to anembodiment;

FIG. 9 illustrates a GUI view according to an embodiment;

FIG. 10 illustrates a GUI view according to an embodiment; and

FIG. 11 illustrates a method of determining a target position accordingto an embodiment.

DETAILED DESCRIPTION

Certain exemplary embodiments are described in greater detail below withreference to the accompanying drawings.

In the following description, the same drawing reference numerals areused for the same elements even in different drawings. The mattersdefined in the description, such as detailed construction and elements,are provided to assist in a comprehensive understanding of exemplaryembodiments. Thus, it is apparent that exemplary embodiments can becarried out without those specifically defined matters. Also, well-knownfunctions or constructions are not described in detail since they wouldobscure exemplary embodiments with unnecessary detail.

Terms such as “part” and “portion” used herein denote those that may beembodied by software or hardware. According to exemplary embodiments, aplurality of parts or portions may be embodied by a single unit orelement, or a single part or portion may include a plurality ofelements.

In exemplary embodiments, an image may include any medical imageacquired by various medical imaging apparatuses such as a magneticresonance imaging (MRI) apparatus, a computed tomography (CT) apparatus,an ultrasound imaging apparatus, or an X-ray apparatus.

Also, in the present specification, an “object”, which is a thing to beimaged, may include a human, an animal, or a part thereof. For example,an object may include a part of a human, that is, an organ or a tissue,or a phantom.

Throughout the specification, an ultrasound image refers to an image ofan object processed based on ultrasound signals transmitted to theobject and reflected therefrom.

As used herein, the term “and/or” includes any and all combinations ofone or more of the associated listed items. Expressions such as “atleast one of,” when preceding a list of elements, modify the entire listof elements and do not modify the individual elements of the list.

FIG. 1 is a block diagram illustrating a configuration of an ultrasounddiagnosis apparatus 100, i.e., a diagnostic apparatus, according to anexemplary embodiment.

Referring to FIG. 1, the ultrasound diagnosis apparatus 100 may includea probe 20, an ultrasound transceiver 110, a controller 120, an imageprocessor 130, one or more displays 140, a storage 150, e.g., a memory,a communicator 160, i.e., a communication device or an interface, and aninput interface 170.

The ultrasound diagnosis apparatus 100 may be of a cart-type or aportable-type ultrasound diagnosis apparatus, which is portable,moveable, mobile, or hand-held. Examples of the ultrasound diagnosisapparatus 100 of a portable type may include a smart phone, a laptopcomputer, a personal digital assistant (PDA), and a tablet personalcomputer (PC), each of which may include a probe and a softwareapplication, but embodiments are not limited thereto.

The probe 20 may include a plurality of transducers. The plurality oftransducers may transmit ultrasound signals to an object 10 in responseto transmitting signals received by the probe 20, from a transmitter113. The plurality of transducers may receive ultrasound signalsreflected from the object 10 to generate reception signals. In addition,the probe 20 and the ultrasound diagnosis apparatus 100 may be formed inone body (e.g., disposed in a single housing), or the probe 20 and theultrasound diagnosis apparatus 100 may be formed separately (e.g.,disposed separately in separate housings) but linked wirelessly or viawires. In addition, the ultrasound diagnosis apparatus 100 may includeone or more probes 20 according to embodiments.

The controller 120 may control the transmitter 113 for the transmitter113 to generate transmitting signals to be applied to each of theplurality of transducers based on a position and a focal point of theplurality of transducers included in the probe 20.

The controller 120 may control an ultrasound receiver 115 to generateultrasound data by converting reception signals received from the probe20 from analog to digital signals and summing the reception signalsconverted into digital signals, based on a position and a focal point ofthe plurality of transducers.

The image processor 130 may generate an ultrasound image by usingultrasound data generated from the ultrasound receiver 115.

The display 140 may display a generated ultrasound image and variouspieces of information processed by the ultrasound diagnosis apparatus100. The ultrasound diagnosis apparatus 100 may include two or moredisplays 140 according to the present exemplary embodiment. The display140 may include a touch screen in combination with a touch panel.

The controller 120 may control the operations of the ultrasounddiagnosis apparatus 100 and flow of signals between the internalelements of the ultrasound diagnosis apparatus 100. The controller 120may include a memory for storing a program or data to perform functionsof the ultrasound diagnosis apparatus 100 and a processor and/or amicroprocessor (not shown) for processing the program or data. Forexample, the controller 120 may control the operation of the ultrasounddiagnosis apparatus 100 by receiving a control signal from the inputinterface 170 or an external apparatus.

The ultrasound diagnosis apparatus 100 may include the communicator 160and may be connected to external apparatuses, for example, servers,medical apparatuses, and portable devices such as smart phones, tabletpersonal computers (PCs), wearable devices, etc., via the communicator160.

The communicator 160 may include at least one element capable ofcommunicating with the external apparatuses. For example, thecommunicator 160 may include at least one of a short-range communicationmodule, a wired communication module, and a wireless communicationmodule.

The communicator 160 may receive a control signal and data from anexternal apparatus and transmit the received control signal to thecontroller 120 so that the controller 120 may control the ultrasounddiagnosis apparatus 100 in response to the received control signal.

The controller 120 may transmit a control signal to the externalapparatus via the communicator 160 so that the external apparatus may becontrolled in response to the control signal of the controller 120.

For example, the external apparatus connected to the ultrasounddiagnosis apparatus 100 may process the data of the external apparatusin response to the control signal of the controller 120, received viathe communicator 160.

A program for controlling the ultrasound diagnosis apparatus 100 may beinstalled in the external apparatus. The program may includeinstructions to perform part of operation of the controller 120 or theentire operation of the controller 120.

The program may be pre-installed in the external apparatus or may beinstalled by a user of the external apparatus by downloading the programfrom a server that provides applications. The server that providesapplications may include a recording medium where the program is stored.

The storage 150 may store various data or programs for driving andcontrolling the ultrasound diagnosis apparatus 100, input and/or outputultrasound data, ultrasound images, applications, etc.

The input interface 170 may receive a user's input to control theultrasound diagnosis apparatus 100 and may include a keyboard, button,keypad, mouse, trackball, jog switch, knob, a touchpad, a touch screen,a microphone, a motion input means, a biometrics input means, etc. Forexample, the user's input may include inputs for manipulating buttons,keypads, mice, trackballs, jog switches, or knobs, inputs for touching atouchpad or a touch screen, a voice input, a motion input, and abioinformation input, for example, iris recognition or fingerprintrecognition, but an exemplary embodiment is not limited thereto.

An example of the ultrasound diagnosis apparatus 100 according to thepresent exemplary embodiment is described below with reference to FIGS.2A, 2B, and 2C.

FIGS. 2A, 2B, and 2C are diagrams illustrating ultrasound diagnosisapparatuses according to an exemplary embodiment.

Referring to FIGS. 2A and 2B, the ultrasound diagnosis apparatus 100 aor 100 b may include a main display 121 and a sub-display 122. At leastone of the main display 121 and the sub-display 122 may include a touchscreen. The main display 121 and the sub-display 122 may displayultrasound images and/or various information processed by the ultrasounddiagnosis apparatus 100 a or 100 b. The main display 121 and thesub-display 122 may provide graphical user interfaces (GUIs), therebyreceiving user's inputs of data to control the ultrasound diagnosisapparatus 100 a or 100 b. For example, the main display 121 may displayan ultrasound image and the sub-display 122 may display a control panelto control display of the ultrasound image as a GUI. The sub-display 122may receive an input of data to control the display of an image throughthe control panel displayed as a GUI. The ultrasound diagnosis apparatus100 a or 100 b may control the display of the ultrasound image on themain display 121 by using the input control data.

Referring to FIG. 2B, the ultrasound diagnosis apparatus 100 b mayinclude a control panel 165. The control panel 165 may include buttons,trackballs, jog switches, or knobs and may receive data to control theultrasound diagnosis apparatus 100 b from the user. For example, thecontrol panel 165 may include a time gain compensation (TGC) button 171and a freeze button 172. The TGC button 171 is to set a TGC value foreach depth of an ultrasound image. Also, when an input of the freezebutton 172 is detected during scanning of an ultrasound image, theultrasound diagnosis apparatus 100 b may keep displaying a frame imageat that time point.

The buttons, trackballs, jog switches, and knobs included in the controlpanel 165 may be provided as a GUI to the main display 121 or thesub-display 122.

Referring to FIG. 2C, an ultrasound diagnosis apparatus 100 c mayinclude a portable device. An example of the ultrasound diagnosisapparatus 100 c of a portable type may include, for example, smartphones including probes and applications, laptop computers, personaldigital assistants (PDAs), or tablet PCs, but an exemplary embodiment isnot limited thereto.

The ultrasound diagnosis apparatus 100 c may include the probe 20 and amain body 40. The probe 20 may be connected to one side of the main body40 by wire or wirelessly. The main body 40 may include a touch screen145. The touch screen 145 may display an ultrasound image, variouspieces of information processed by the ultrasound diagnosis apparatus100 c, and a GUI.

FIG. 3 is a block diagram of a structure of an ultrasound imagingapparatus 300 according to an embodiment.

According to an embodiment, the ultrasound imaging apparatus 300includes a probe 310, a processor 320, and a display 330.

The probe 310 includes a plurality of transducers and transmitsultrasound signals to an object and detects echo signals reflected fromthe object. The probe 310 may correspond to the probe 20 of FIG. 1.

The processor 320 controls all operations of the ultrasound imagingapparatus 300. The processor 320 may be implemented as one or moreprocessors. The processor 320 receives ultrasound signals from the probe310 to reconstruct an ultrasound image. Ultrasound signals generated bythe probe 310 undergo predetermined signal processing through abeamformer, an amplifier, an analog-to-digital converter (ADC), etc.,and are then transmitted to the processor 320. The processor 320 mayexecute an instruction or command stored in a memory to perform aspecific operation.

The display 330 displays ultrasound images and predetermined data. Thedisplay 330 displays a GUI view provided by the ultrasound imagingapparatus 300. The display 330 may correspond to the display 140 of FIG.1.

FIG. 4 is a diagram for explaining a process of transferring an embryoto the uterus, according to an embodiment.

An embryo transfer procedure for transferring an embryo 430 to a uterus410 has been widely used for in vitro fertilization (IVF). Real-timemedical images are necessarily required to protect a patient duringembryo transfer and improve a success rate of the embryo transfer.Ultrasound imaging is widely used in medical procedures because it mayoffer medical images of an object in real-time.

During an embryo transfer procedure, a procedure tool 420 containing theembryo 430 is inserted into a patient's body, and the embryo 430 in theprocedure tool 420 is then transferred to a proper position in theuterus 410. To increase a success rate of embryo transfer, it iscritical to transfer the embryo 430 to the proper position in the uterus410. To find the proper position in the uterus 410 for embryo transfer,the medical personnel captures an image of a region of the uterus 410via the probe 310 to examine a real-time medical image 440.

However, because an ultrasound image obtained as a real-time medicalimage is usually a two-dimensional (2D) ultrasound image, there is alimitation in acquiring information about a three-dimensional (3D)structure within the uterus. Furthermore, when only a 2D ultrasoundimage is used, there is a limitation that parameters obtainable from a3D medical image cannot be utilized.

According to embodiments of the disclosure, an embryo transfer processmay include determining a target position for embryo transfer by using a3D medical image of the uterus 410, identifying the target position in areal-time ultrasound image, and displaying information about the targetposition on the display 330. Thus, a success rate of transfer of theembryo 430 to the uterus 410 may be increased.

According to embodiments of the disclosure, the real-time medical image440 may be a 2D or 3D ultrasound image. Although embodiments of thedisclosure are described herein with respect to an example in which thereal-time medical image 440 is a 2D ultrasound image, the scope of thepresent application is not limited thereto.

FIG. 5 is a flowchart of a method of controlling an ultrasound imagingapparatus, according to an embodiment.

Operations of a method of controlling an ultrasound imaging apparatusaccording to the disclosure may be performed by an ultrasound imagingapparatus including a probe and a processor. The disclosure will bedescribed with respect to an embodiment in which the ultrasound imagingapparatus 300 performs a method of controlling an ultrasound imagingapparatus. 300 is hereinafter used as a reference numeral collectivelydenoting ultrasound imaging apparatuses according to embodiments of thedisclosure. Thus, embodiments described with respect to the ultrasoundimaging apparatus 300 may be applied to a method of controlling theultrasound imaging apparatus 300. Conversely, embodiments described withrespect to a method of controlling the ultrasound imaging apparatus 300may be applied to embodiments described with respect to the ultrasoundimaging apparatus 300. Although it has been described that methods ofcontrolling an ultrasound imaging apparatus according to embodiments ofthe disclosure are performed by the ultrasound imaging apparatus 300,embodiments are not limited thereto, and the methods may be performed byvarious types of ultrasound imaging apparatus.

Referring to FIG. 5, first, the ultrasound imaging apparatus 300 obtainsa 3D medical image of the uterus (S502). The ultrasound imagingapparatus 300 may include a memory storing 3D medical images.

According to an embodiment, the ultrasound imaging apparatus 300 obtainsa 3D medical image of the uterus from an external apparatus. Accordingto an embodiment, the 3D medical image may be an ultrasound image.According to another embodiment, the 3D medical image may be a medicalimage of a different imaging modality than ultrasound, such as a CT orMR image. The ultrasound imaging apparatus 300 may include acommunicator via which a 3D medical image is obtained from an externalapparatus. The ultrasound imaging apparatus 300 may provide a mode foracquiring a 3D medical image obtained from an external apparatus, a modefor loading the 3D medical image, etc.

According to another embodiment, the ultrasound imaging apparatus 300may obtain a 3D ultrasound image of the uterus via the probe 310.According to an embodiment, a 3D ultrasound image and a 2D ultrasoundimage may be obtained using different types of the probe 310. For thispurpose, a 3D probe and a 2D probe may be respectively used to acquire a3D ultrasound image and a 2D ultrasound image. According to anotherembodiment, the ultrasound imaging apparatus 300 may acquire both 3D and2D ultrasound images via a single probe. In this case, the probe 310 maybe a probe for performing both 3D and 2D imaging, and for example, maycorrespond to a 3D vaginal probe, a 2D matrix probe, etc. The ultrasoundimaging apparatus 300 may provide an imaging process and a mode forsequentially performing 3D imaging and 2D imaging.

According to an embodiment, a 3D medical image and a real-timeultrasound image may be obtained using a probe capable of performing 3Dimaging. The real-time ultrasound image may correspond to a 3Dultrasound image. For example, a 3D medical image and a real-time 3Dultrasound image may be obtained using a 2D matrix probe.

Next, the processor 320 determines a target position for embryo transferby using the 3D medical image (S504).

The processor 320 recognizes an anatomical structure of the uterus inthe 3D ultrasound image. For example, the processor 320 recognizes inthe 3D medical image an anatomical structure such as an ovary, afallopian tube, an endometrium, a perimetrium, a junction zone, etc. Forrecognition, predetermined segmentation may be performed on the 3Dmedical image.

Furthermore, the processor 320 extracts parameter values related toembryo transfer from the 3D medical image. Parameter values related toembryo transfer may include at least one of a maximal implantationpotential (MIP) point, an endometrial thickness, a blood flow amount, ajunction zone location, a uterine shape, an implantation location, auterine location, an endometrial location, and a location of a lesion(e.g., myoma or adenomyoma), or a combination thereof. At least oneparameter value related to embryo transfer may be calculated in a 3Dspace. Furthermore, at least one parameter value may be determined foreach position in the 3D space. For example, at least one parameter valuemay be matched to at least some coordinates in the 3D space or to eachcoordinate therein. A map representing a parameter value in the 3D spacemay be generated for a specific parameter value corresponding to atleast some positions or coordinates in the 3D space.

A target position for embryo transfer may be determined based on atleast one of an anatomical structure of the uterus and at least oneparameter value related to the embryo transfer or a combination thereof.The processor 320 may determine a target position for embryo transferwhere a success rate of embryo transfer is high, based on at least oneof an anatomical structure of the uterus and at least one parametervalue related to embryo transfer. The target position may be determinedas a specific point or region. The target position may be determined inthe 3D space within the uterus.

Then, the processor 320 obtains a real-time ultrasound image of theuterus by using the probe 310 (S506).

The processor 320 matches the real-time ultrasound image that is a 2Dultrasound image to the target position and at least one parameter valueacquired based on the 3D ultrasound image. The processor 320 mayregister the 2D ultrasound image with the 3D medical image to identifythe target position in the 2D ultrasound image and match at least oneparameter value to the 2D ultrasound image.

Various techniques may be used for registration between the 2Dultrasound image and the 3D medical image.

According to an embodiment, the processor 320 may register the 2Dultrasound image with the 3D medical image by using image registration.For example, the processor 320 may perform image registration by usinganatomical features or by matching edges in the 2D ultrasound image withedges in the 3D medical image.

According to another embodiment, the processor 320 may register the 2Dultrasound image with the 3D medical image by using a detection valuedetected by a sensor. For example, the probe 310 may include a positionsensor for detecting a position of the probe 310, an accelerometer orgyro sensor for detecting its orientation, or a combination thereof. Theprocessor 320 may identify a plane corresponding to the 2D ultrasoundimage in the 3D medical image by using position information ororientation information detected by the sensor and may match the 2Dultrasound image to the plane identified in the 3D medical image.

Subsequently, the processor 320 displays on the display 330 thereal-time ultrasound image and information about the target position(S510). The target position may be displayed in the 2D ultrasound image.According to an embodiment, the processor 320 may display a 2Dultrasound image and information about the target position on a GUIview, together with at least one parameter value.

Furthermore, the processor 320 may recognize a procedure tool in the 2Dultrasound image and provide a guide for moving the procedure tool tothe target position based on a position of the procedure tool. Forexample, the processor 320 may provide information about the position ofthe procedure tool or information about how to move the procedure toolto the target position.

When the real-time ultrasound image does not correspond to a plane wherethe target position is located, the processor 320 may provide a guidefor moving the probe 310 to image the plane corresponding to the targetposition.

As the procedure tool moves, the processor 320 may update a guide formovement of the procedure tool and a guide for movement of the probe310.

FIG. 6 illustrates a 3D medical image 610 and a 2D ultrasound image 620according to an embodiment.

The 3D medical image 610 and the 2D ultrasound image 620 are capturedimages of the same patient's uterus 410. The 3D medical image 610 andthe 2D ultrasound image may be captured at adjacent time points. Forexample, the 3D medical image 610 and the 2D ultrasound image 620 may becaptured on the same day.

The 2D ultrasound image 620 may correspond to a predetermined plane 630in a 3D space. The predetermined plane 630 corresponding to the 2Dultrasound image 620 may change in real-time according to movement ofthe probe 310. According to an embodiment, the ultrasound imagingapparatus 300 may provide the 2D ultrasound image 620 together with aposition of the probe 310 in the 3D space and information about thepredetermined plane 630 corresponding to the 2D ultrasound image 620.

FIG. 7 illustrates an MIP point according to an embodiment.

A target position for embryo transfer may be determined based on atleast one of an anatomical structure of the uterus and at least oneparameter value related to the embryo transfer or a combination thereof.The processor 320 may create, based on the anatomical structure of theuterus and the at least one parameter value, a 3D transfer probabilitymap showing a good position for transfer and store the 3D transferprobability map. At least one parameter may include an endometrialthickness, a blood flow amount, a junction zone location, a uterineshape, an implantation location, a uterine location, an endometriallocation, an MIP point, a uterine morphological pattern, etc.

A score indicating a good position for embryo implantation ishereinafter referred to as an implantation score. An implantation scoreis a value representing an implantation success rate.

The processor 320 may score a good position for embryo implantationaccording to an endometrial thickness. For example, when the endometrialthickness does not exceed 7 mm, an implantation success rate decreases.

The processor 320 may determine an implantation score according to theamount of blood flow. The processor 320 may determine an implantationscore based on the amount of blood flow in an angiography image. As theamount of blood flow increases, an implantation success rate isconsidered to be higher.

The processor 320 may determine an implantation score according to athickness, a sharpness, and contraction/relaxation extent of a junctionzone. As the junction zone has a more uniform thickness, the probabilityof successful implantation is considered to be higher. As the junctionzone is connected more smoothly without being cut off, the probabilityof successful implantation is considered to be higher.

The processor 320 may determine an implantation score based on uterinecontraction and relaxation When there is no uterine contraction, animplantation success rate is considered to be higher. Furthermore, whenuterine contraction and relaxation are to a smaller extent, theprobability of successful implantation is considered to be higher.

The processor 320 may determine an implantation score according to auterine shape and an implantation location. An optimal position forimplantation may be in a triangular region at the center of the uterus,and the processor 320 may determine an implantation score according towhether a location for implantation is within the triangular region Thetriangular region at the center of the uterus is determined as an MIPpoint 716. According to an embodiment, a point where lines 712 and 714connected along two uterine tubes intersect each other is determined asthe MIP point 716, and use of the MIP point 716 or a predeterminedregion including the MIP point 716 is considered to achieve a highprobability of successful implantation.

The processor 320 may determine an implantation score according to amorphological pattern of the uterus. Homogeneity of a morphologicalpattern may indicate a high probability that adenomyoma or myoma existsin the uterus, and an implantation success rate may decrease due to thepresence of such a lesion.

By applying an implantation score determined based on a plurality ofparameter values to a predetermined equation, the processor 320 mayobtain an evaluation value indicating an implantation success rate basedon a resultant value obtained using the equation. The evaluation valuemay be calculated according to a position in the uterus. The evaluationvalue may be calculated in various forms such as a final implantationsuccess rate, a final implantation score, etc.

The processor 320 may create a 3D transfer probability map by mapping anevaluation value determined as described above according to a positionin the uterus and store the 3D transfer probability map. The 3D transferprobability map may be displayed on 2D or 3D image data based on anevaluation value. The 3D transfer probability map may represent anevaluation value at each position in the uterus by using graphicalindicators such as a color, a graph, etc.

FIG. 8 illustrates a GUI view according to an embodiment.

According to an embodiment, the processor 320 may display on the display330 a GUI view that provides both a 3D medical image 610 and a 2Dultrasound image 620.

According to an embodiment, the GUI view may provide a first indicator870 showing a position in a 3D space corresponding to the 2D ultrasoundimage 620. The first indicator 870 may be provided together with asecond indicator (875) indicating the probe 310. The first indicator 870may be updated in response to a change in a position of the probe 310.The processor 320 may calculate a position in the 3D space correspondingto the 2D ultrasound image 620 according to movement of the probe 310and update the first indicator 870 based on the calculated position inthe 3D space.

According to an embodiment, the GUI view may show a uterine boundary 810in the 2D ultrasound image 620. The uterine boundary 810 refers to anouter boundary of the uterus. According to an embodiment, the processor320 may automatically recognize and display the uterine boundary 810 inthe 2D ultrasound image 620. According to another embodiment, theprocessor 320 may determine the uterine boundary 810 based on a userinput received via an input interface, and display the uterine boundary810 in the 2D ultrasound image 620.

According to an embodiment, the GUI view may show an endometrialboundary 820 in the 2D ultrasound image 620. According to an embodiment,the processor 320 may automatically recognize and display theendometrial boundary 820 in the 2D ultrasound image 620. According toanother embodiment, the processor 320 may determine the endometrialboundary 820 based on a user input received via the input interface anddisplay the endometrial boundary 820 in the 2D ultrasound image 620.

According to an embodiment, the GUI view may provide characteristicsrelated to a junction zone surrounding the endometrial boundary 820. Forexample, the GUI view may provide, on the 2D ultrasound image 620,information 830 about the amount of blood flow in a junction zone.

According to an embodiment, the GUI view may provide a contractionindicator 840 indicating the extent of contraction of the junction zone.The contraction indicator 840 may indicate the extent of contraction ofthe uterus. A success rate of embryo transfer may be increased byperforming the embryo transfer when there is no uterine contraction.According to an embodiment, the contraction indicator 840 may bedisplayed as a circle and shown in a first color (e.g., green) whencontraction is not detected or in a second color (e.g., red) whencontraction is detected. According to an embodiment, the processor 320may acquire information about contraction of the uterus based on the 2Dultrasound image 620. For example, the processor 320 may acquireinformation about the contraction of the uterus based on movement of atleast of the uterine boundary 810 and the endometrial boundary 820 or acombination thereof. According to another embodiment, the processor 320may receive and acquire information about contraction of the uterus froman external apparatus.

According to an embodiment, the GUI view may provide information about atarget position 850. The processor 320 may determine the target position850 based on the above-described criteria and display an indicatorindicating the target position 850 in the 2D ultrasound image 620.

According to an embodiment, the GUI view may provide guide information860 indicating a direction in which a procedure tool is to move to reachthe target position 850. For example, the guide information 860 may beprovided in the form of a vector or an arrow indicating a direction inwhich the procedure tool has to move.

FIG. 9 illustrates a GUI view according to an embodiment.

According to an embodiment, the GUI view may provide an imaging positionindicator 930 indicating information about whether the 2D ultrasoundimage 620 obtained via the probe 310 is being captured of a targetposition. The processor 320 may calculate information about a distancebetween the target position and the 2D ultrasound image 620 beingcaptured and provide the information about the distance via a GUI. Forexample, when the target position is in a plane corresponding to 910 cand the 2D ultrasound image 620 obtained via the probe 310 deviates fromthe plane corresponding to 910 c but is in a plane corresponding to 910a, 910 b, or 910 d, the target position may not be properly imaged.According to an embodiment, the imaging position indicator 930 mayinclude a first sub-indicator 932 corresponding to a position of a planecorresponding to the target position and second sub-indicators 934 a,934 b, 934 c, and 934 d corresponding to a position of a planecorresponding to the 2D ultrasound image 620 captured by the probe 310.When a plane corresponding to the 2D ultrasound image 620 moves from 910a to 901 b to approach 910 c including the target position, the secondsub-indicator 934 a may be changed to the second sub-indicator 934 b toindicate that an imaging position corresponding to the 2D ultrasoundimage 620 is close to the target position. When the plane correspondingto the 2D ultrasound image 620 moves from 910 b to 910 d to be in adifferent direction than 910 c including the target position, the secondsub-indicator 934 c may be changed to the second sub-indicator 934 d bychanging the type of a line of the second sub-indicator 934 b.

FIG. 10 illustrates a GUI view according to an embodiment.

According to an embodiment, the GUI view may show a procedure toolindicator 1010 a or 1010 b indicating a position of a procedure tool.The procedure tool indicator 1010 a or 1010 b may be displayed in only a2D ultrasound image 620, only a 3D medical image 610, or both the 2Dultrasound image 620 and the 3D medical image 610. The processor 320 maydetect the procedure tool in the 2D ultrasound image 620. The 2Dultrasound image 620 may show the procedure tool being imaged. Theprocessor 320 may provide the procedure tool indicator 1010 a bydisplaying a boundary of the imaged procedure tool in the 2D ultrasoundimage 620. The processor 320 may determine a position of the proceduretool in the 3D medical image 610 based on the position of the proceduretool detected in the 2D ultrasound image 620 and display the proceduretool indicator 1010 b in the 3D medical image 610 based on thedetermined position of the procedure tool.

FIG. 11 illustrates a method of determining a target position accordingto an embodiment.

According to an embodiment, the processor 320 may determine a targetposition for embryo transfer based on a 3D medical image by using a deepneural network (DNN) processor 1110. The DNN processor 1110 may beincluded in the processor 320 or may be provided as a separate processorin the ultrasound imaging apparatus 300 or in an external apparatus suchas a server. According to an embodiment, the DNN processor 1110 receivesa 3D medical image to output a target position for embryo transfer.According to another embodiment, the DNN processor 1110 may receive a 3Dmedical image and a 2D ultrasound image. According to anotherembodiment, the DNN processor 1110 may receive a 3D medical image andadditional information (e.g., uterine contraction information) acquiredfrom the external apparatus. According to another embodiment, the DNNprocessor 1110 may output a target position and at least one parametervalue related to embryo transfer.

The DNN processor 1110 may include a pre-trained DNN. For example, theDNN processor 1110 may include a neural network structure such as aconvolutional neural network (CNN) or a recurrent neural network (RNN),or a combination of various neural network structures.

Embodiments of the disclosure may be implemented through non-transitorycomputer-readable recording media having stored thereoncomputer-executable instructions and data. The instructions may bestored in the form of program code, and when executed by a processor,generate a predetermined program module to perform a specific operation.Furthermore, when being executed by the processor, the instructions mayperform specific operations according to the embodiments.

According to embodiments of the disclosure, a success rate of embryotransfer to the uterus may be increased.

Furthermore, according to embodiments of the disclosure, userconvenience may be increased and a guide may be provided to the userduring embryo transfer to the uterus.

While embodiments of the disclosure have been particularly shown anddescribed with reference to the accompanying drawings, it will beunderstood by those of ordinary skill in the art that various changes inform and details may be made therein without departing from essentialcharacteristics or the spirit and scope of the disclosure as defined bythe appended claims. The disclosed embodiments should be considered in adescriptive sense only and not for purposes of limitation.

What is claimed is:
 1. An ultrasound imaging apparatus comprising: aprobe configured to transmit ultrasound waves and detect echo signals; adisplay; and at least one processor configured to generate an ultrasoundimage based on the echo signals, wherein the at least one processor isfurther configured to: obtain a three-dimensional (3D) medical image ofa uterus, recognize an anatomical structure of the uterus in the 3Dmedical image, acquire a value of at least one parameter based on the 3Dmedical image, create a 3D transfer probability map including anevaluation value indicating an implantation success rate at eachposition in the uterus based on the anatomical structure of the uterusand the value of the at least one parameter, determine a target positionfor optimal embryo placement based on the 3D transfer probability map,obtain a real-time ultrasound image of the uterus that is atwo-dimensional (2D) ultrasound image, register the real-time ultrasoundimage with the 3D medical image, determine whether the real-timeultrasound image corresponds to a plane where the target position isincluded, and control the display to display the real-time ultrasoundimage and an indicator that guides for moving the probe to obtain thereal-time ultrasound image corresponding to the plane where the targetposition is included.
 2. The ultrasound imaging apparatus of claim 1,wherein the at least one processor is further configured to match thevalue of the at least one parameter acquired based on the 3D medicalimage to the real-time ultrasound image, and control the display todisplay information about the value of the at least one parameter in thereal-time ultrasound image.
 3. The ultrasound imaging apparatus of claim1, wherein the at least one parameter comprises at least one of anendometrial thickness, a blood flow amount, a junction zone location, auterine boundary location, an endometrial location, a myoma location,and an adenomyoma location, or a combination thereof.
 4. The ultrasoundimaging apparatus of claim 1, wherein the probe comprises at least onesensor including at least one of a position sensor, a gyro sensor, andan accelerometer, and wherein the at least one processor is furtherconfigured to register the 3D medical image with the real-timeultrasound image based on a signal detected by the at least one sensor.5. The ultrasound imaging apparatus of claim 1, wherein the at least oneprocessor is further configured to control the display to display anindicator representing a position of a plane corresponding to thereal-time ultrasound image with respect to the 3D medical image.
 6. Theultrasound imaging apparatus of claim 1, wherein the 3D medical imageand the real-time ultrasound image are respectively obtained byapparatuses using different imaging modalities.
 7. The ultrasoundimaging apparatus of claim 1, wherein the at least one processor isfurther configured to control the display to display information aboutthe evaluation value at a position of at least a portion of the uterus.8. The ultrasound imaging apparatus of claim 1, wherein the at least oneprocessor is further configured to control the display to display anindicator representing a distance between a plane corresponding to thereal-time ultrasound image and the plane where target position isincluded.
 9. The ultrasound imaging apparatus of claim 1, wherein thereal-time ultrasound image shows at least one procedure tool insertedinto the uterus, and wherein the at least one processor is furtherconfigured to control the display to display the 3D medical image andthe real-time ultrasound image and provide, on the 3D ultrasound image,information about a position of the at least one procedure tool detectedin the real-time ultrasound image.